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1 .  INTRODUCTION 


A.  Introduction 


This  report  is  the  documentation  for  the  contract,  "Study  of 
a  Heterogenous  Distributed  Microcomputer  Network  Using  Measured 
Data  and  Analytical/Simulation  Models,"  (Contract  No. 
DAAK70-79-D-0087)  done  as  a  joint  project  between  Georgia 
Institute  of  Technology  (Georgia  Tech)  and  the  Army  Institute  for 
Research  in  Management  and  Information  Sciences  (AIRMICS)  over  the 
one-year  period  of  June  15,  I960  to  June  15,  1981. 

In  the  past  decade  or  so,  a  number  of  computer-communication 
networks  have  come  into  being  that  allow  the  joint  sharing  of  the 
resources  of  a  number  of  computers.  The  advent  of  microcomputers 
has  accelerated  the  potential  gains  to  be  achieved  from 
distributed  computer  networks,  but  at  the  same  time  has 
necessitated  some  changes  in  the  ground  rules  for  designing 
computer  networks  and  has  introduced  new  design  problems  arising 
from  the  attempt  to  add  significantly  Improved  features. 

This  study  has  been  concerned  with  exploring  the 
characteristics  of  a  network  with  potential  application  In 
management  information  systems  using  both  analytic/s iaulat ion 
models  (developed  in  the  study)  and  measured  data  obtained  from 
the  network  models.  The  network  used  has  been  constructed  in  the 
School  of  Electrical  Engineering  at  Ceorgia  Tech  with  support  from 
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AIRMICS  and  is  described  in  detail  later  in  this  report.  A 
significant  feature  of  the  proposed  program  has  been  the  approach 
of  dealing  with  both  experimental  data  from  the  physical  network 
and  results  obtained  from  analytical/simulation  models. 

B.  Objective 

The  completed  study  has  been  directed  toward  an  enhanced 
understanding  of  heterogenous,  distributed  microprocessor 
networks.  Monitor  equipment  has  been  studied,  installed  and  used 
to  obtain  experimental  data  on  the  operation  of  the  network  and 
analytical/simulation  models  have  been  developed.  The 
analytical/simulation  models  were  tailored  to  describe  as 
accurately  as  possible  the  operation  of  the  actual  network. 

The  results  of  the  study  are  models  which  accurately  describe 
network  behavior.  These  models,  which  depend  explicitly  on 
well-defined  parameters,  provide  the  facility  to  investigate 
alternative  designs  and  to  predict  network  behavior  for  many  types 
of  inputs,  such  as  would  be  specified  for  particular  management 
information  systems. 

The  study  required  four  tasks: 

Task  1^:  Design  and  Implementation  of  Non-lntrusive  Monitoring 
Equipment 

The  first  task  of  this  study  synthesized  a  limited, 
cost-effective  collection  of  hardware  and  software  to  obtain  the 


data  required  to  validate  analytic/simulation  models  and 
characterize  network  performance.  The  heart  of  the  hardware 
monitor  is  the  "monitoring  memory",  a  two  port  memory  which  Is  on 
both  the  monitor's  bus  and  the  node  microprocessor's  bus,  and 
which  is  Implemented  non-intruslvely .  By  correctly  programming 
the  monitor  processor,  any  function  in  the  node  may  be  monitored. 

Task  2:  Design  and  Implementation  of  Traffic  Generating  Equipment 

The  second  task  measured  the  behavior  of  the  network  under 
various  controlled  stimuli.  To  accomplish  this,  equipment  was 
developed  which  could  generate  network  traffic  messages.  This 
involved  both  hardware  and  software  development.  The  hardware  was 
designed  to  allow  the  traffic  generating  machine  to  communicate 
with  both  the  network  processors  and  the  monitoring  processors. 
Real  time  software  was  necessary  to  drive  the  hardware  and 
actually  implement  an  experiment. 

Task  3^:  Development  of  Analytical/Simulation  Models 

The  third  task  was  the  development  of  the 
analytical/simulation  model  for  the  AIRMICS/Georgia  Tech 
experimental  network.  The  model  represents  the 
computer-communication  system  and,  to  some  extent,  the  host 
microcomputers  as  a  network  of  queues.  Of  particular  Interest 
were  the  following  performance  measurements:  1}  end-to-end  delay 
for  the  individual  elements  of  the  communication  network,  2) 
channel  utilizations,  3)  throughput  for  all  elements  of  the 
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network  and  hosts,  and  4)  throughput  In  terms  of  job  completions 
for  the  entire  system.  Analytical  analysis  was  used  to  predict 
the  network  delays  as  a  function  of  system  loading. 

Task  4:  Evaluation  and  Test  of  the  Model  with  the  Experimental 
Network 

The  fourth  task  was  the  prediction  of  results  of  various 
amounts  of  network  loading  based  on  the  theoretical  models.  The 
variables  accounted  for  by  the  queueing  network  model  were 
measured  in  the  experimental  network  under  controlled  conditions. 
The  results  of  such  measurements  were  then  compared  to  computed 
results  from  the  model  and  the  model  refined  to  minimize 
differences.  The  results  of  no  load  and  minimal  load  conditions 
were  verified  experimentally;  however,  hardware  limitations 
prevented  the  testing  of  average  and  full  load  conditions. 
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2.  Mathematical  Models 


For  the  AIRMICS/Georgia  Tech  Network 


2.1  Introduction 

This  section  of  the  report  presents  and  discusses  queueing 
models  for  elements  of  the  AIRMICS/Georgia  Tech  network.  It  also 
reviews  several  approaches  that  can  be  used  in  analytical  studies  of 
network  configurations. 

As  discussed  in  detail  elsewhere  in  this  report,  the  network 
consists  of  a  maximum  of  five  switching  nodes  which  can  be  connected 
in  a  variety  of  ways  to  form  a  packet  switched  network.  In 
operation,  the  network  interconnects  host  computers,  terminals  and 
other  devices  which  may  be  distributed  over  a  relatively  large 
geographical  area. 

The  interest  in  much  of  the  present  study  is  centered  on  the 
performance  of  the  network  as  measured  by  such  quantities  as 
throughput  and  delay,  which  are  distinctive  properties  of  the 
network.  Although  these  quantities  depend  on  the  nature  of  the 
Inputs  and  outputs  of  the  network,  they  can  be  considered  to  a  large 
extent  independent  of  specific  details  of  the  interconnected  hosts, 
etc.  in  analyzing  or  designing  a  network. 
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Queueing  theory  models  have  been  used  almost  exclusively  in 
obtaining  mathematical  relations  for  throughput  and  delay.  There  is 
a  considerable  amount  of  literature  on  queueing  theory  as  a 
mathematical  discipline,  (see  for  example  [1],  [2],  and  [3]),  and  in 
the  last  decade  there  have  been  numerous  and  significant  studies  of 
computer  networks,  as  modeled  by  interconnected  systems  of  queues, 
[4J ,  ( 5 1 ,  [6].  The  literature  describing  the  matching  of 
mathematical  models  to  specific  physical  systems,  however,  is 
relatively  sparse  [7]. 

The  material  concerning  mathematical  models  and  performance 
analysis  in  this  Section  is  organized  as  follows:  Section  2.2  gives 
basic  queueing  models  for  network  components.  Several  parameters  of 
the  queueing  models,  namely,  buffer  capacity,  the  effect  of 
acknowledgements,  input  and  output  statistics,  and  errors  are 
discussed  in  Section  2.3. 

Section  2.4  discusses  methods  for  analysis  with  the  queueing 
models.  A  number  of  techniques  based  on  different  types  of 
approximations  are  discussed  in  this  Section.  Section  2.5  gives  a 
concise  review  of  the  network  models  and  solution  techniques  and 
Section  2.6  lists  references.  An  Appendix  concludes  the  section. 

2.2  Queueing  Models  for  the  Network  Components 


The  basic  elements  of  the  network  are  the  Switching  Nodes  and 
the  Connnecting  Lines  or  Channels.  Connected  to  the  network  will  be 
Host  Computers,  Terminals,  and  possibly  other  devices  such  as  Disk 
Memories,  etc. 

The  interest  in  the  present  study  is  in  modeling  the  network  and 
those  aspects  of  the  devices  connected  to  the  network  which  will 
affect  the  planned  experiments.  This  will  involve  modeling  Host 
Computers,  used  in  an  elementary  data  processing  role,  as  well  as  the 
basic  network  elements. 

The  network  switching  nodes  are  microcomputers  with  Buffer 
Memories  and  four  I/O  lines  controlled  by  the  Nodal  CPU.  Arriving 
messages  queue  for  the  use  of  the  Node  CPU.  The  Node  CPU  controls 
instructions  for  error  detection,  storing  incoming  messages  in 
buffers,  sending  acknowledgement  messages  to  the  node  originating  the 
message,  and  choosing  the  output  node  and  route  to  destination.  The 
time  to  process  these  instructions  is  called  "nodal  processing  time" 
and  is  typically  small  enough  to  be  neglected  in  a  queueing  model. 

Messages  routed  to  an  output  node  by  the  CPU  must  queue  in  an 
output  Buffer  for  the  use  of  the  output  channel.  The  "service  time" 
of  the  output  channel  is  determined  by  its  line  capacity  using  the 
relation  "service  time"  equals  message  length  divided  by  line 
capacity.  This  service  time  is  Included  in  queueing  models  to 
account  for  the  channel.  Typically  the  "Propagation  Time,"  or  time 
for  one  bit  to  travel  from  the  sending  to  the  receiving  terminal,  is 
small  enough  to  be  negligible. 
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Figure  2.1  gives  a  basic  model  of  the  Switching  Node  and  Output 
Channels,  neglecting  nodal  processing  time  and  propagation  time. 
Models  such  as  that  in  Figure  2.1  are  discussed,  for  example,  by 
Kleinrock  [8],  Wong  [4],  and  Samari  [6]. 

For  use  in  the  Experiments  discussed  below,  the  Host 
Minicomputers  can  be  modeled  by  the  single  queue  shown  in  Figure 
2.2a.  A  slightly  more  general  model  with  an  additional  queue  and 
processor  FM  (file  manager)  is  hown  in  Figure  2.2b.  The  additional 
processor  determines  whether  a  received  message  requires  local 
processing  or  is  intended  for  another  Host.  More  general  models  for 
Minicomputer  Hosts  are  discussed  by  many  authors,  see  for  example  [9] 
and  [10]. 

2.3  Parameters  of  the  Queueing  Models 

2.3.1  Buffer  Capacity:  The  capacity  of  the  Buffer  Pool  at 
each  Switching  Node  is  finite,  being  3200  bytes  total.  In  the 
physical  system,  the  finite  buffer  capacity  leaves  open  the 
possibility  that  some  messages  will  be  lost  because  there  will  be  no 
buffer  in  which  to  store  them.  The  possibility  of  occasional  lost 
messages  could  be  Indicated  in  the  model  of  Figure  2.1  by  adding  a 
dotted  "lost  traffic”  branch  to  every  incoming  channel. 

To  make  a  first  order  model  tractable  for  analytic  solution,  it 
is  convenient  to  assume  that  buffers  are  infinite.  This  assumption 
is  not  absolutely  essential  and  finite  buffer  models  are  discussed 
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FIGURE  2.1.  BASIC  MODEL  OF  SWITCHING  NODE  AND  OUTPUT  CHANNELS. 


below. 


2.3.2  Acknowledgements:  The  network  is  structured  so  that 
positive  acknowledgements  are  exchanged  between  Nodes.  If  a  message 
is  lost  or  received  in  error  it  is  not  acknowledged  and,  after  a 
specified  "time-out",  the  message  is  retransmitted. 

Acknowledgement  messages  are  ten  bytes  in  length  and  are 
transmitted  between  Nodes  on  paths  directed  in  the  reverse  direction 
from  the  message  traffic.  In  general,  acknowledgements  will  queue 
with  other  traffic  traveling  in  its  direction  between  the  Nodes. 

Acknowledgements  impact  the  queueing  models  in  two  ways.  The 
nodal  CPU  must  direct  the  storing  (actually  the  retention  in  storage) 
of  replicas  of  each  transmitted  message  until  an  acknowledgement  is 
received  at  which  time  the  retained  message  is  "written  over".  This 
requires  some  part  of  the  nodal  processing  time,  but  since  the  total 
nodal  processing  time  is  usually  negligible  with  respect  to  other 
effects,  this  increment  to  nodal  processing  time  is  typically  ignored 
in  the  models. 

The  second  impacc  on  queueing  is  more  important. 
Acknowledgement  traffic  must  be  added  to  ocher  traffic  and  accounted 
for  in  determining  queue  sizes. 


2.3.3  Input/Output  Traffic  Statistics:  In  an  analysis  with 
queueing  models,  the  statistical  distributions  of  arriving  message 
traffic  must  be  known  or  assumed.  The  same  comment  applies  to  the 
time  required  for  message  transmission  over  the  connecting  channels 
(i.e. ,  service  times). 

With  respect  to  arriving  traffic,  the  statistical  distribution 
will,  obviously,  depend  on  particular  applications.  There  is, 
however,  very  little  literature  describing  measured  distributions  of 
message  traffic.  The  data  that  exists  is  not  in  conflict  with 
assuming  the  most  tractable  process,  namely  Poisson,  for  arriving 
traffic. 

Now  consider  service  times.  The  number  of  bits  per  second 
processed  over  a  communication  channel  is  constant  at  the  rated 
capacity  of  the  channel,  during  the  time  messages  are  queued  and 
ready.  The  length  of  messages,  however,  can  vary  so  that  the  time  to 
process  complete  messages  can  be  variable.  As  with  arriving  message 
distributions,  the  statistical  distributions  of  message  lengths  will 
vary  with  the  application  and  little  data  on  typical  distributions  is 
available. 

Logical  choices  for  the  distribution  function  of  message  lengths 
include  the  geometric  distribution,  and  in  fact,  the  (deterministic) 
constant  message  length.  An  exponential  distribution  for  message 
lengths  is  often  assumed  to  give  a  tractable  analytical  model. 
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2.3.4  Random  Errors:  The  network  deals  with  errors  through  the 
use  of  an  error  detecting  code.  After  a  packet  has  been  transferred 
between  two  nodes,  the  overhead  bits  for  error  detecting  are 
evaluated  and  a  positive  acknowledgement  is  sent  back  to  the 
originating  node  if  the  packet  has  been  received  correctly.  If  the 
received  packet  is  not  correct,  no  acknowledgement  is  sent  and, 
after  a  time  out,  the  packet  is  retransmitted. 

An  exact  model  of  the  network  would  have  to  account  for  the 
finite  probability  of  retransmission  occuring  due  to  random  bit 
error.  For  a  well  designed  network  operating  over  short  lines, 
however,  the  probability  of  bit  error  is  low  and  therefore, 
retransmissions  are  not  an  important  factor  in  network  operation. 
They  will  not  be  included  in  the  models  to  be  discussed  below. 


2.4  Network  Studies  with  Queueing  Models 

To  model  a  specific  network  configured  from  the  basic  components 
of  the  AIRMICS/Georgia  Tech  network,  the  basic  queueing  models  must 
be  combined  into  a  network  of  interconnected  queues.  The  Interest  in 
the  model  centers  on  the  ability  to  obtain  throughputs  and  delays 
for  the  composite  network  and  to  relate  these  quantities  to  basic 
network  parameters  such  as  input  traffic,  line  capacities,  packet 
lengths,  etc. 
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Two  basic  approaches  are  available  for  studying  the  models; 


analytic  methods  and  simulation  methods.  Both  approaches  have 
advantages  and  disadvantages. 

Analytic  methods  have  the  very  desirable  property  of  yielding 
explicit  relations  between  the  variables  and  making  direct  design 
procedures  and  design  trade-offs  possible.  Unfortunately,  to  make 
analytical  methods  feasible,  a  number  of  approximations  must 
typically  be  made,  and  accuracy  of  approximation  becomes  a  serious 
question. 

Simulation  methods,  on  the  other  hand,  especially  direct  "Monte 
Carlo"  simulations  do  not  require  the  degree  of  approximation  of 
models  used  for  analytic  work.  The  shortcoming  of  simulation 
methods,  however,  lies  in  the  fact  that  they  do  not  yield  explicit 
relations  between  the  variables.  Instead,  every  set  of  numerical 
values  for  the  parameters  must  be  used  in  separate,  complete, 
calculations.  To  obtain  desired  statistics,  it  is  usually  necessary 
to  obtain  many  "runs”  tor  a  single  set  of  parameters  and  employ 
statistical  estimation  to  obtain  the  desired  results. 

Given  the  properties  of  the  two  basic  approaches  to  model 
analysis,  it  seems  reasonable  to  use  both  in  any  extensive  studies. 
Analytical  results,  and  the  explicit  relations  which  they  provide, 
would  seem  to  give  essential  background  or  "first  cut"  information 
and  thus  they  are  obtained  first.  The  present  study  is  limited  to 
the  analytical  methods. 
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2.4.1  Models  Which  Assume  Independence  Of  Queues; 

When  a  message  path  through  a  network  must  pass  through  one  or  more 
Switching  Nodes,  the  queueing  model  becomes  a  tandem  connection  of 
queues  with  the  output  of  one  feeding  into  the  input  of  another. 
Analytic  methods  for  tandem  queues  are  severely  restricted  and 
tractable  results  require  the  equivalent  of  the  following  assumptions 
[11],  [12] : 

1.  The  external  network  arrivals  follow  a  Poisson  process 

2.  Each  time  a  message  enters  a  new  node  it  is  assigned  a  new  length 
selected  from  exponential  distribution;  all  message  classes  have 
the  same  message  e  • » tii  distribution 

3.  The  routing  algorithm  is  fixed. 

4.  All  buffers  have  infinite  capacity. 

5.  The  queuing  discipline  is  first  come  -  first  served. 

Kleinrock  [8],  for  example,  discusses  these  assumptions  in  detail  and 
shows  that  when  they  apply,  each  node  can  be  analyzed  independently 
as  an  M/M/1  queue*.  The  total  average  network  delay  is  then  a  simple 
weighted  sum  of  the  independ» ntly  calculated  nodal  delays. 

In  spite  of  the  severity  of  these  assumptions,  several  authors 
[13],  [8]  have  noted  that  in  treating  many  effects  and  for  most 
networks,  the  answers  are  sufficiently  accurate  for  engineering 
purposes. 


*M/M/1  is  a  common  notation  for  Poisson 
arrivals,  exponential  service,  and  one  server. 
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The  basic  M/M/1  equation  for  the  average  time  spent  waiting  for 


and  using  the  ith  channel  is  given  by: 


Ti  = 


uCr 


(2.1) 


where  \ is  the  traffic  rate  in  the  ith  channel,  C.  is  its 
capacity,  and  1/ p  is  the  average  message  length. 


This  equation  can  be  adapted  to  the  case  of  combined  data 
traffic  and  control  traffic.  The  average  waiting  time  is  due  to  both 
types  of  traffic,  while  the  service  time  is  due  only  to  the  data 
traffic.  This  yields: 


Ti  =  u‘crxi 


pC. 


(2.2) 


where  1/p1  is  the  average  message  length  of  all  packets  and  1/p  is 
the  average  length  of  data  packets. 


If  nodal  processing  time  is  denoted  as  K,  and  propagation  time 
on  the  ith  channel  as  ,  then  Kleinrock  gives  the  total  average 
message  delay,  T,  for  the  whole  network  as: 


T 


m 

K  +  L 


i=1 


S.  r xi/u'Ci  .  1 


+  Pi  +  K 


(2.3) 


where  Y  is  the  total  traffic  entering  the  network  in 
messages/second,  and  m  is  the  number  of  inter-nodal  links. 


The  simplicity  of  (2.3)  makes  it  possible  to  obtain  approximate 
results  in  a  straightforward  manner  even  for  complicated  network 
structures. 


2.4.2  Models  with  Finite  Buffer  Space:  Although  the  tractable 
models  of  Section  2.4.1  will  solve  many  network  problems,  there  are 
several  effects  which  are  directly  attributed  to  finite  buffer  space. 
These  effects  are  lost  messages  due  to  full  buffers,  congestion, 
deadlock  and  other  flow  problems.  It  is,  of  course,  desirable  to 
have  models,  amenable  to  analytic  solution,  which  can  approximate 
these  effects. 

Several  approaches  from  the  literature  will  be  reviewed  after 
some  comments  on  the  general  problems. 

Loss  of  packets  can  occur,  for  example,  if  a  packet  attempts  to 
enter  a  node  in  which  all  available  buffer  space  for  the  required 
output  channel  is  full.  The  packet  will  be  blocked  from  entering  the 
node  and  thus  will  be  stored  in  the  previous  node  until  buffer  space 
becomes  available,  (thereby  propagating  congestion  back  through  the 
network),  or  may  be  dropped  after  being  locally  acknowledged, 
(thereby  requiring  retransmission  from  the  originating  node).  The 
general  effect  of  packet  loss  and  other  congestion  in  the  network  is 
to  cause  a  drop  in  throughput  as  illustrated  in  Figure  2.4.1. 

Deadlock  is  a  more  severe  flow  control  problem  in  which  the 
throughput  for  a  particular  link  goes  to  zero.  This  may  occur  when 
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•CONGESTION 


ITERISTIC  WITHOUT  FLOW  CONTROL. 


two  nodes  wishing  to  communicate  have  all  the  buffers  allocated  to 


the  required  link  full.  Reassembly  lock-up  is  another  finite 
storage  problem  which  may  occur  in  the  destination  switching  node 
where,  in  many  networks,  message  reassembly  takes  place. 

Finite  buffer  queueing  models  for  switching  nodes  have  been 
developed  and  used  with  considerable  success  in  buffer  management  and 
local  flow  control  problems.  The  models  retain  the  assumptions 
listed  in  Section  2.4.1  for  the  most  tractable  models,  with  the 
exception  of  the  infinite  buffer  (assumption  4).  Irland  [14]  and  Lam 
[15]  both  have  analyzed  models  for  a  single  node  with  finite  buffers. 
Their  results  are  summarized  as  follows:  Consider  a  Poisson  packet 
arrival  stream  with  average  rate  X  (the  offered  load)  arriving  at  a 
node,  capable  of  holding  at  most  N  packets,  see  Figure  2.4.2.  Let  C 
be  the  transmission  rate  of  the  single  output  channel  and  U  the 

average  packet  length.  These  incoming  packets  are  blocked  (and 

dropped  from  the  network)  with  probability  P  ,  the  probability  of 

B 

all  the  buffers  being  occupied,  so  that  the  throughput,  Y  ,  is  given 
by: 

Y  =  X  (1-PB)  .  (2.4) 


The  effect  of  nodal  blocking  »t  the  next  stage  of  a  tandem  connection 

is  to  reduce  the  effective  channel  capacity  to  C(l-P  )  assuming  an 

B 

identical  blocking  probability.  Under  this  assumption,  the  queueing 
model  for  a  channel  becomes  effectively  a  finite  M/M/1  queue  for 
which  the  blocking  probability  is  given  by: 
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p  -  UbeM 

D  "  ,  N+l 

I  ~P 


(2.5) 


where 


P  = 


pC(l-Pn) 


(2.6) 


Combining  (2.4),  (2.5),  and  (2.6)  gives  the  equation  for  a 

throughput-offered  load  characteristic  such  as  that  of  Figure  2.4.1. 


In  a  more  general  model  of  a  node,  such  as  that  of  Figure  2.4.3 
there  are  a  number  of  output  channels  to  other  nodes  and  perhaps  a 
link  to  a  local  host  as  well  as  an  input  buffer.  Such  a  node  also 
deals  with  locally  generated  traffic,  traffic  coming  from  similar 
switching  nodes  (often  termed  transit  traffic),  and  positive 
acknowledgements  and  timeouts.  Irland  [14]  showed  that  restricted 
buffer  sharing  policies  improve  the  throughput  unde"'  ingest.'" 
conditions.  Lam  [15]  developed  an  algorithm  for  determining  the 
nodal  buffer  requirements  so  as  to  minimize  the  loss  probabilities 
and  thereby  balance  the  traffic  among  the  output  channels.  Their 
work  was  extended  by  Lam  and  Reiser  [16]  who  showed  that  input  buffer 
limits  could  be  used  effectively  as  a  form  of  congestion  control. 

Penotti  and  Schwartz  [17]  considered  a  number  of  finite  storage 
nodes  in  tandem  along  a  virtual  circuit  such  as  that  shown  in  Figure 
2.4.4.  They  model  the  effect  of  external  arrivals  by  reducing  the 
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time-out  ACK 


link  capacity  by  the  external  packet  arrival  rate;  i.e.  for  link  i, 
the  effective  capacity  becomes: 


u 


ei 


“icrxi 


(2.7) 


where  X-j  is  the  external  packet  arrival  rate  to  link  i.  This  is 
equivalent  to  reducing  the  network  of  Figure  2.4.4  to  that  of  Figure 
2.4.5.  If  (2.4)  is  used  to  express  the  effective  service  rate  for 
the  ith  link,  the  result  is: 

"('’“ei  •  (2'8) 


Where  P  is  the  blocking  probability  for  the  ith  link. 

B  l 

The  overall  blocking  probability  for  the  complete  circuit  of 
Figure  2.4.5  can  be  evaluated  iteratively  working  backwards  from  the 
final  node.  The  overall  throughput  is  then  found  to  be: 

Y  =  »0O-pB,)  (2.9) 

where  X  is  the  load  offered  to  the  virtual  circuit  by  the  source. 
0 

Simulation  results  for  local  control  have  shown  good  agreement 
with  this  approximate  model.  Rudin  [18]  has  extended  the  work  of 
Pennotti  and  Schwartz  to  study  the  phenomenon  of  reduced  throughput 
with  increased  offered  load. 
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A  more  powerful  approach  to  modeling  finite  buffer  storage 
networks  than  those  dlscused  above  involves  the  use  of  a  Markov  chain 


to  model  the  complete  network.  This  method  potentially  removes  most 
of  the  reotrictions  listed  in  Section  2.4.1.  Some  work  has  been 
found  in  the  literature  using  this  method.  For  example,  a  recent 
paper  [19]  discusses  a  two  node  model  which  gives  a  six  or  eight 
dimensional  state  probability  vector.  This  model  incorporates 
restrictions  to  the  buffer  and  window  sizes  as  well  as 
differentiating  between  local  and  "foreign"  traffic;  acknowledgements 
are  also  included  in  the  model.  The  author  uses  sparse  matrix 
algorithms  to  attack  the  large  dimensional  matrices  which  must  be 
inverted. 

Although  the  paper  cited,  and  a  few  others  provide  an  approach 
to  the  Markov  chain  method,  they  do  not  result  in  practical  methods. 

Since  there  is  a  need  for  more  accurate  methods  of  dealing  with 
the  finite  buffer  problem,  some  time  was  spent  on  the  present  project 
in  attempting  to  develop  an  alternate  Markov  chain  approach.  This 
work  is  discussed  in  the  next  Section. 


2.4.3  A  New  Approach  for  Finite  Buffer  Models  Using 
Markov  Chains;  If  one  continuous  time  Markov  chain  is  used  to  model 
the  complete  network,  each  state  in  the  chain  will  have  a  value 
equal  to  the  number  of  packets  (queued  and  being  served)  on  a 
particular  channel  of  the  network.  Since  the  total  storage  at  each 
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node  is  finite,  the  Markov  chain  is  finite  valued,  although  in 
general,  the  number  of  states  is  extremely  large.  The  general 

technique  to  solve  such  a  Markov  chain  is  to  write  down  the  local 
balau.e  equations  [1]  and  to  solve  them  for  the  state  probabilities 
P(np  n7,  .  .  .  i^),  i.e.,  the  probability  of  having  n^  packets  in 

queue  1,  .  .  .  nn  packets  in  queue  n.  Such  a  method  is  tedious  and 

costly,  as  the  inversion  of  an  extremely  large  matrix  is  required. 

An  alternative  graphical  method  of  solving  for  the  state 
probabilities  in  a  Markov  chain  is  proposed  using  a  state  Reduction 
Method  developed  by  Liu  [20].  Liu's  method  is  developed  for  discrete 
time  Markov  chains  and  gives  the  mean  first  passage  time  tcom  one 
specified  state  to  another.  Work  on  the  present  project  has  extended 
the  general  method  to  continuous  time  Markov  chains  as  discussed  in 
an  Appendix.  From  the  mean  first  passage  times,  the  state 

probabilities  and  other  parameters  of  interest,  such  as  average  delay 
time,  throughput  and  average  number  of  packets  stored  at  each  node, 
have  been  evaluated. 

To  illustrate  the  method  devised,  consider  a  queueing  model  with 
the  following  characteristics: 

1.  The  network  is  "closed”  so  that  a  fixed  number  of  messages  are 
circulating. 

2.  Service  times  are  exponentially  distributed. 

3.  There  is  a  maximum  to  the  number  of  messages  in  the  network  such 
as,  for  example,  might  be  imposed  by  window  flow  control. 
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Assumption  3  insures  that  the  storage  used  is  finite  and  hence  both 
finite  and  infinite  buffers  give  the  same  results.  A  specific 
example  from  the  above  class  is  used  to  illustrate  the  method. 


Consider  a  closed  queueing  network  with  3  queues  in  tandem  and 
with  2  messages  circulating  such  as  shown  in  Figure  2.4.6.  All 
queues  can  be  assumed  to  have  infinite  capacity. 

The  messages  can  be  distributed  Throughout  the  network  in  six 
possible  ways,  so  that  the  Markov  chain  has  six  states.  Each  of  the 
service  times  is  exponentially  distributed  with  means  1/y^  ,  1 /y  , 
and  1/  respectively.  The  continuous  time  Markov  chain  can  be 
represented  by  the  transition  rate  diagram  of  Figure  2.4.7. 
Kleinrock  [1]  and  other  texts  on  Markov  chains  show  how  this  diagram 
is  developed. 

In  the  diagram,  the  three  digit  numbers  in  the  "state  circles" 
identify  the  states  by  giving  the  number  of  messages  in  queue  1,  the 
number  of  messages  in  queue  2,  and  the  number  of  messges  in  queue  3 
in  the  order  stated.  These  queue  occupancies  are  denoted  n^  ,  n^  , 
and  respectively.  For  the  network  defined  above, 

n^+  i\2+  n^=  2.  (2.10) 


The  standard  approach  to  determining  the  steady  state 
probabilities  is  to  solve  the  matrix  equation,  [1], 


xQ  =  0 
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(2.11) 


subject  to  the  constraint 

6 

£  11 1  =  1  *  (2.12) 

i=l 

In  (2.11),  it  “  [  ,  713  »  »  *5  ,  itg  ]  is  a  vector  of  state 

probabilities,  tr^.  ,  and  Q  is  the  transition-rate  matrix 


For  example,  if  *3,  "2,  "1,  the  matrix  equation  (2.11) 

yields  the  solution 
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w4  =  P01 1  ~ 


nl  =  p020  "  9/85 

n2  =  pno  =  6/85  *5  =  P101  =  12/>85 

^3  =  p200  ”  4//85  ^6  =  P002  =  36/85  • 

For  a  large  number  of  states,  such  as  would  result  for  a  more 
reasonable  number  of  messages  circulating  than  the  two  assumed, 
solving  the  set  of  linear  equations  given  by  (2.11)  would  become 
costly,  since  matrix  inversion  is  required- 

The  alternate  method  investigated  involves  three  steps:  (1) 

use  of  the  Liu  Reduction  Method  to  find  the  mean  first  passage  time 
between  states,  (2)  determination  of  the  mean  recurrence  times  for  a 
subset  of  states,  and  (3)  determination  of  the  state  probabilities. 

Carrying  out  step  (1)  for  discrete-time  Markov  chains  requires 

converting  the  non-weighted  chain  (for  example,  the  chain  of  Figure 
2.4.7),  to  a  weighted  one.  For  continuous-time 

Markov  chains,  the  extension  of  Liu's  work  discussed  in  the 
Appendix  gives  the  necessary  cost  function  for  state  i  as  1/q^  , 
where  q  ^  ■  -q  ^  ,  and  q^  is  the  ith  diagonal  element  of  the  Q 
matrix.  The  appropriate  transition  probabilities  from  state  1  to 
state  j  for  the  cost-weighted  chain  are: 
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lVV  '  * j  • 

The  chain  with  the  appropriate  cost  function  is  shown  in  Figure 
2. A. 8.  To  continue  with  the  example,  the  cost-weighted  chain  of 
Figure  2.4.8  can  be  reduced  to  two  states  using  the  Reduction  Method. 
Steps  in  the  reduction  procedure  are  shown  in  Figures 
2.4.9a  -  2.4.9f;  Figure  2.4.9g  gives  the  final  result. 

It  is  a  property  of  the  cost-weighted  chains  constructed  in  the 
reduction  method  that  at  any  stage  of  the  reduction  process  the  cost 
of  a  state  is  equal  to  the  mean  first  passage  time  from  that  state  to 
the  group  of  states  remaining  at  that  stage  of  the  reduction  process. 
Thus  from  Figure  2.4.9g,  the  mean  first  passage  time  from  state  2  to 
state  3  is  27/4,  while  that  from  state  3  to  state  2  is  1/3. 

The  second  step  in  the  alternative  method  is  to  calculate  mean 
recurrence  times  for  the  states  required  from  the  basic  relation, 

m, 

’j  Mjj = ' +  £  qjiMu  ’ 
i/j 

where  Mjj  is  the  mean  recurrence  time  of  state  j. 

In  the  example,  if  is  the  only  state  probability  required, 
it  can  be  computed  from  M33  .  The  mean  recurrence  time  M33  is 
computed  as  follows 
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state  assignment  number 


FIGURE  2.4.8.  COST-WEIGHTED  MARKOV  CHAIN. 


FIGURE  2.4.9.a 


2/5 


FIGURE  2.4.9.b 
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FIGURE  2.4.9c 


FIGURE  2.4.9d 


FIGURE  2.4.9e 


FIGURE  2.4.9f 


FIGURE  2.4.9g 


It  should  be  noted  that,  In  general,  (2.14)  could  require  a 

number  of  M. .  for  computing  M..  .  In  such  a  case  additional  steps 
ij  J3 

would  be  required  in  the  reduction  procedure. 

A  special  case  of  some  interest  is  the  class  of  networks  for 
which  "product-form"  solutions  apply  [8].  The  example  under 

consideration,  being  a  closed  network,  is  in  this  class. 

Product-form  solutions  express  all  state  probabilities  in  terms  of 
the  same  parameters. 
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For  example,  the  probability  of  any  state,  identified  as 


{n  1  n  2  n  j) ,  Is  given  by 


P 


3 

{state  =  G(N)  IT  X. 

irl 


(2.16) 


where 


N  =  n-j  +n2+h3  =  2  ; 


and  the  X  are  solutions  of 
l 


X-j  =  v*2  ^2 

u2  X2  =  u3  X3 


(2.17) 


and  G(N)  is  a  normalization  constant.  Equation  (2.16)  can  be  used  to 
express  all  X^  in  terms  of  one  chosen  Xj  ,  such  as  X3  .  For  the 
example,  this  gives 

(T  o  n.  ru  n,  o  h,  n- 

state  =  G(2)  (f  X2)  1  (X2)  *  (2  X2)  J  =  G(2)X2^(2/3)  '2 

P.18) 


Knowledge  of  one  state  probability  is  sufficient  to  determine 
G(2)  X  2^  and  hence,  all  state  probabilities.  To  continue  with  the 
example  above,  the  state  reduction  method  yields: 


*  P  ^state  20o| 


=  4/85. 


Use  of  this  value  gives 


G(2)X22  =  9/85 


and 


P  {state  nin2n3^ 


■Is  l|>n,(2> 


It  should  be  pointed  out  that  in  addition  to  the  method 
developed,  other  computational  algorithms  are  discussed  in  the 
literature  for  direct  calculation  of  important  system  parameters  for 
product-form  queueing  networks,  [A].  Specific  mention  should  be  made 
of  the  convolutional  algorithm  of  Buzen  and  the  mean  value  analysis 
algorithm  proposed  by  Reiser  and  Lavenberg. 


2.5  Discussion 

Modeling  and  mathematical  analysis  of  a  computer-communication 
network  must,  to  some  extent,  be  tailored  to  particular  applications 
and  desired  results.  This  Section  reviews  the  approaches  which  can 
be  used. 

Queueing  models  are  almost  always  used  for  performance  analysis. 
Section  2.1  gives  references  to  a  number  of  papers  covering 
background  material.  Section  2.2  gives  specific  queueing  models  for 
the  two  major  components  of  the  AIRMICS/Georgia  Tech  network. 


Operating  conditions  for  the  queueing  networks  are  discussed  in 
Section  2.3  under  the  heading  of  "Parameters  for  the  Queueing  Model.” 

Approaches  to  studies  with  queueing  models  are  reviewed  in 
Section  2.4.  It  is  pointed  o'*t  that  both  analytic  and  simulation 
studies  of  queueing  models  can  be  made. 

Generally  speaking,  simulation  studies  are  necessary  for  fairly 


accurate 

results  if  the  network 

has 

tandem 

queues 

and  if  the 

" product - 

form"  assumptions  are  not 

justified. 

Such 

studies  are, 

however, 

expensive  in  computer 

time 

and  do 

not 

give  explicit 

relations 

between  variables.  Simulation 

studies 

would 

seem  to  be 

appropriate  when  all  but  a  few  parameters  are  specified,  (Including 
input  message  distributions  and  message  length  distributions),  and 
accurate  curves  are  desired  for  the  relationships  between,  at  most, 
several  variables. 

Analytic  studies  tend  to  be  approximate  but  have  the  advantage 
of  yielding  explicit  relations  between  the  variables.  The  report 
gives  conditions  such  that  a  very  tractable  product-form  solution  can 
be  obtained. 

One  restriction  on  the  product-form  solution  is  the  assumption 
of  Infinite  storage  at  the  nodes.  This  assumption  is  obviously  not 
valid  and  its  removal  is  necessary  in  order  to  study  certain 
important  effects.  The  report  reviews  work  on  this  problem  from  the 
literature  and  also  gives  the  results  of  a  new  method  developed  on 
the  project. 


With  respect  to  analytical  studies,  it  is  recommended  that  first 
cut  approximate  solutions  be  obtained  from  assumptions  yielding  the 
most  tractable  product-form  solutions.  Such  solutions  are  used  in 
Section  3  in  analyzing  controlled  experiments  with  the  network.  In 
spite  of  their  approximate  nature,  there  is  considerable  evidence  for 
adequate  accuracy  from  the  very  tractable  models. 

If  blocking  and  other  problems  asociated  with  finite  buffers  are 
to  be  studied,  the  choice  of  approach  is  less  clear.  All  of  the 
methods  reviewed,  including  the  one  developed  on  the  project,  have 
restrictions.  For  issues  of  buffer  management  and  flow  control  at  a 
single  node,  the  methods  of  Irland,  Lam  and  Reiser  seem  to  be 
attractive. 

On  the  other  hand,  for  complete  networks  the  approximate  method 
of  Penotti  and  Schwartz  seems  to  be  the  most  tractable  available. 
Their  method  is  not  general  and  its  restrictions  have  not  been 
investigated  by  the  authors  of  this  report. 

The  most  accurate,  method  of  working  with  finite  storage  at  the 
network  nodes  uses  a  Markov  chain  type  model.  This  type  of  model  is 
accurate,  but  typical  parameter  values  lead  to  an  extremely  large 
number  of  states  in  the  model.  Conventional  approaches  require  the 
inversion  of  very  large  matrixes.  The  method  developed  on  this 
project  attacks  the  problem  of  large  numbers  of  states  by  a  state 
reduction  technique.  The  method  could  have  advantages  when  it  is 
necessary  to  model  more  than  one  node  of  the  network  in  considering 
blocking,  deadlock  or  some  other  type  of  degradation.  The  use  of 


mean  first  passage  time  from  some  reference  state  to  some  other 
state,  or  group  of  states  where  some  type  of  degradation  occurs  can 
act  as  a  criterion  for  network  design. 
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For  discrete-time  irreducible  recurrent  Markov  chains,  it  is 
fairly  straightforward  [3]  to  show  that  the  mean  first  passage  times 
( "k  * ’3*^1  are  related  by  the  set  of  linear  equations: 


M(k  ’ 1 *£  pij  V  for  3,1  J',k 


(A.  1) 


where  ^  ijj  are  ttle  cransit?on  probabilities  of  the  .Markov  chain. 
The  mean  recurrence  time  for  any  state  k  is  likewise  given  by: 


Mkk  ’  1  +  £  »kj  Mjk 


(A. 2) 


\  similar  set  of  equations  holds  for  cont inuous-t ime  irreducible 
recurrent  Markov  chains,  namely 


qiMik  =  ’  *  £  '’ij  Mjk  for  a"  m 


(A.  3) 


and 
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"kls'* 


\)  MJk  for  all  k 


where  q  —  ,  i*j 


consequence  of 


is  the  transition  rate 
the  total  transition 
the  definitions: 


from  state  i  to  state 
rate  out  of  state  i. 


and 
As  a 


=  I  *1J 


(A.  5) 


Lqij  =  l  alii, 

j 


(A.  6) 


The  transition  rate  matrix  ot  the  continuous  tir..e  fiarkov  chain  is 
denoted  ^  and  is  given  by: 


Q=U,j]  i,j=0,  1.... 


To  prove  (A. 3)  and  (A. 4)  rigorously  is  quite  difficult,  some 
fundamental  theorems  from  renewal  theory  are  involved  (see  for 
example,  (22]  and  (23j). 
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it  (A. 3)  is  rewritten  as: 


3.  CONTROLLED  EXPERIMENTS 


FOR  STUDYING  MATHEMATICAL  MODELS 


3 . 1  Int  roduct ion 

Section  2  of  this  report  presents  queueing  models  of  the 
components  of  the  AIRMICS /Georgia  Tech  network  and  discusses  several 
sets  of  assumptions  which  lead  to  alternative  analytic  solutions. 
The  purpose  of  this  section  is  to  describe  several  experiments  to  be 
made  with  elementary  network  configurations  under  controlled 
conditions.  These  experiments  are  tailored  to  provide  tests  of  the 
various  mathematical  models  for  assessing  their  accuracy. 

The  experiments  are  regarded  as  a  first  step  in  evaluating  the 
models  under  conditions  which  attempt  to  focus  on  one  or  two  effects 
at  a  time.  Results  of  the  experiments  are  expected  to  lead  to 
refinements  of  the  models  and  possibly  refinements  in  the  physical 
network.  It  is  anticipated  that  later  work  will  be  directed  toward 
studies  of  more  typical  network  configurations  under  realistic 
conditions  using  both  the  models  and  the  physical  network. 

The  material  in  this  section  will  be  presented  in  a  format  which 
describes  an  experiment,  gives  a  queueing  model  of  the  network 
configuration,  and  finally  gives  an  analytic  solution  for  measurable 
quantities,  such  as  average  message  delay,  as  a  function  of  network 
parameters. 
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Several  Appendices  to  this  section  present  details  of  the 


analytic  solutions  which  were  not  covered  in  Section  2. 


3-2  Experimental  Network  Configurations 

The  proposed  experiments  configure  the  network  in  two  ways:  as 
a  loop  of  two  nodes  and  as  a  loop  of  three  nodes  as  shown  in  Figure 
3.2.1.  For  both  configurations  traffic  is  generated  by  a  traffic 
generating  computer  and  for  the  experiments  each  message  is  one 
packet  in  length. 

Two  experiments  involve  "open  networks"  in  the  sense  that 
messages  arrive  at  the  network  and  then  pass  through  it  without 
returning. 

Two  experiments  involve  "closed  networks"  for  which  a  fixed 
number  of  messages  circulate  around  the  network.  In  practice  this 
could  be  accomplished  by  connecting  the  output  terminals  of  the 
network  back  to  the  input  terminals  with  a  fixed  number  of  messages 
stored  in  the  network  buffers  prior  to  actuating  the  system.  For  the 
experiments,  it  is  more  convenient  to  achieve  a  closed  network  by 
programming  the  message  generating  computer  to  input  a  new  message  to 
the  network  each  time  a  message  exits  from  the  network. 

In  addition  to  the  four  experiments  described  above,  a  fifth 
experiment  is  designed  to  investigate  the  effects  of  finite  storage 
using  an  open  configuration. 
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The  two  experimental  configurations  are  chosen  for  the  following 
reasons.  The  network  is  designed  so  that  positive  acknowledgements 
are  generated  when  a  packet  is  correctly  received  in  transmission 
between  nodes.  Thus,  for  example,  if  a  packet  is  transmitted  from 
Node  A  to  Node  B  and  received  correctly,  an  acknowledgement  packet  is 
sent  from  Node  B  back  to  Node  A. 

The  two-node  configuration  chosen  is  the  simplest  configuration 
for  which  acknowledgement  traffic  combines  and  queues  with  data 
traffic  for  use  of  the  links.  This  is  apparent  by  considering 
Figure  3.2.1a.  Data  traffic  from  Node  A  to  Node  B  traverses  the  top 
member  of  the  full  duplex  link.  Upon  correct  receipt  at  Node  B,  the 
data  packets  are  "turned  around"  with  negligible  delay  and 
transmitted  from  Node  B  to  Node  A.  But  acknowledgements  are 
generated  for  each  correctly  received  packet  and  the  acknowledgements 
are  also  sent  from  Node  B  to  Node  A.  Similar  reasoning  shows  that 
acknowledgement  packets  combine  with  data  packets  in  going  from  Node 
A  to  Node  B. 

The  three-node  configuration  was  chosen  as  a  configuration  for 
which  acknowledgement  packets  do  not  combine  with  data  packets. 
Examination  of  Figure  3.2.1b  shows  that  for  the  three-node 
configuration,  acknowledgement  packets,  for  example,  will  flow  over 
the  lower  link  from  Node  B  to  Node  A  while  the  data  packets,  which 
generated  the  acknowledgements ,  flow  from  Node  B  to  Node  C  over  the 
lower  link.  Similar  consideration  of  the  other  links  shows  that  data 
and  acknowledgement  packets  remain  separate  on  these  links  also. 
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The  traffic  generating  computer  can  control  several  features  of 


the  generated  traffic,  namely: 

1.  The  packet  length  distribution 

2.  The  mean  packet  length,  1/y 

3.  The  packet  arrival  rate,  x 

4.  The  distribution  of  the  interarrival  times  to  the  first  node. 

Exponential  length  distributions  are  required  for  exact  adherence  to 
the  requirements  for  the  tractable  M/M/1  “ lndependant “  analytical 

model.  Fixed  length  packets  are  possibly  easier  to  generate  and,  in 
many  cases,  provide  results  which  can  be  adequately  approximated  by 
the  tractable  models.  Both  types  of  packets  are  used  in  the 
experiments.  Mean  packet  length  and  packet  arrival  rate  are 

parameters  in  the  experiments  described. 

For  the  experiments,  the  measured  quantities  are  different  delay 
times,  all  of  which  are  measured  with  respect  to  the  first  bit  in  the 
packet.  Total  average  message  delay  is  measured  as  the  average  time 
required  for  packets  to  travel  from  t.ie  traffic  generator  through  the 
network  and  back  to  the  traffic  generating  computer.  The  average  is 
taken  over  a  large  number  of  packets.  Nodal  delay  is  measured  as  the 
average  transit  time  for  packets  from  entry  into  a  particular  node  to 
arrival  at  the  next  destination  node. 

Several  assumptions  are  made  in  obtaining  the  analytical  models 
for  the  experiments.  First  the  "independence"  assumption  is  used  and 
this  requires  that  packet  lengths  be  selected  independently  from  an 
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exponential  distribution  at  each  node.  This  assumption  can  be 
satisfied  at  the  first  node,  where  messages  enter,  but  is  only  an 
approximation  at  other  nodes. 

Infinite  buffers  will  be  assumed.  This  is  an  assumption  for 
open  networks  that  is  approximately  true  if  the  average  number  of 
packets  queueing  at  a  given  node  is  considerably  less  than  the  finite 
buffer  size.  For  closed  networks  the  approximation  is  valid  if  the 
number  of  messages  circulating  is  less  than  the  finite  buffer 
capacity  at  each  node. 

Figure  2.1  of  Section  2  gives  a  relatively  complete  queueing 
network  for  a  switching  node.  The  processing  time  for  the  CPU  to 
transfer  packets  out  of  the  input  line  buffer  is  short  compared  to 
the  time  for  transferring  packets  out  of  the  nodal  buffer  over  the 
line  to  an  adjacent  node.  Thus,  it  can  be  assumed  that  input  line 
buffers  can  be  neglected,  as  such,  and  their  effect  included  with 
nodal  processing  time.  Using  this  assumption,  each  node  can  be 
represented  by  a  single  queue  for  each  output  line  in  a  queue  model. 

Finally,  in  general,  transmission  over  a  line  has  an  associated 
propagation  delay  which  is  typically  on  the  order  of  tens  of 
milliseconds  per  thousand  miles.  Clearly  such  delays  are  negligible 
for  the  very  short  times  used  in  the  experiments. 


3.3  Experiment  One 


This  Experiment  uses  the  three  node  configuration  of  Figure 
3.2.1b.  The  traffic  generator  is  set  up  for  operation  as  an  open 
network  with  packet  length  and  packet  arrival  rate  as  parameters. 
Average  total  message  delay  and  average  nodal  delays  are  measured. 

A  queueing  model  for  the  network  configured  for  this  experiment 
(and  using  the  assumptions  stated)  is  given  in  Figure  3.3.1.  The 
average  nodal  delay,  T^  ,  for  a  typical  node,  assuming  exponentially 
distributed  message  lengths,  can  be  expressed  as 

Ti  ■  +  K  • 

Where  K  is  the  nodal  processing  time  for  the  particular  node.  The 
total  average  message  delay,  T,  through  the  whole  network  can  be 
obtained  by  summing  the  four  nodal  delays  to  obtain 

T  ■  *  K*  *  "d  *  2KC  • 

For  purposes  of  comparison,  the  average  nodal  delay,  assuming  a 
constant  packet  length,  is  given  by 


This  result  is  derived,  for  example,  by  Klelnrock,  [1]. 
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In  presenting  curves  for  total  average  message  delay  and  average 
nodal  message  delay,  It  Is  convenient  to  normalize  the  expressions 
given  above  to  obtain 

nodal  delay:  (T^-K)  = 

total  delay:  (J  -  -  Kg  -  2KC)  = 

nodal  delay,  constant  message  length:  (T.-K)  * 

I  l-P  yC 

where 

K  =  Ka  or  Kg  or  <c 
p  =  X/uC 

The  normalized  average  nodal  delay  curves  for  both  exponentially 
distributed  and  constant  message  lengths  are  given  in  Figure  3.3.2, 
while  the  normalized  average  total  delay  curve  is  given  in  Figure 
3.3.3. 

For  reference,  unnormalized  curves  for  average  total  message 
delay  are  given  in  Figures  3.3.4  and  3.3.5  for  C  ■  1200  bits/sec., 

KA  *  kb  ’ 

and  2000  bits.  A  curve  for  average  nodal  delay  with  both  fixed  and 
exponentially  distributed  message  lengths  is  given  in  Figure  3.3.6 
for  01200  bits/sec.,  K  assumed  to  be  zero  and  an  average  message 
length  of  500  bits. 

« 


Kq  assumed  to  be  zero,  and  message  lengths  of  500 
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NUMBER  OF  MESSAGES  PER  SECOND 
FIGURE  3.3.4.  AVERAGE  MESSAGE  DELAY  VERSUS  MESSAGE/SEC.  FOR 
MESSAGE  LENTTH  =  500  BITS,  C  =  1200 BITS/SEC.,  K‘K,! 


NUMBER  OF  MESSAGES  PER  SECOND 

FIGURE  3.3.5.  AVERAGE  MESSAGE  DELAY  VERSUS  MESSAGES/SEC.  FOR 
MESSAGE  LENGTH  =  2000  BITS,  C  -  1200  BITS/SEC.,  K'K,  ’ 


exponentially 

distributed 


FIGURE  3.3.6.  AVERAGE  NODAL  DELAY  VERSUS  MESSAGES/SEC.  FOR 
MESSAGE  LENGTH  =  500  BITS,  C  =  1200  BITS/SEC.,  K  =  K, 


FIGURE  3.3.2.  AVERAGE  NODAL  DELAY  (NORMALIZED  UNITS)  VERSUS 
MESSAGES/SEC.  (NORMALIZED  UNITS). 


NUMBER  OF  MESSAGES  PER  SEC.,  X 


FIGURE  3.3.3.  TOTAL  AVERAGE  MESSAGE  DELAY  (NORMALIZED  UNITS)  VERSUS 
MESSAGES/SEC.  (NORMALIZED  UNITS). 


3.4  Experiment  Two 


This  experiment  configures  the  network  into  the  two-node 
configuration  shown  in  Figure  3.2.1a.  As  vi th  Experiment  One,  this 
Experiment  is  with  an  open  network,  and  traffic  generation  conforms 
with  this  requirement.  Packet  length  and  packet  arrival  rate  are 
parameters  and  average  total  message  delay  and  average  nodal  delay 
are  the  measured  quantities. 

A  queueing  model  for  the  configuration  is  given  in  Figure  3.4.1. 


Note  that  the 

traffic 

labeled  in 

the 

Figure  accounts 

for 

acknowledgements  as 

well  as 

data  traffic. 

It 

is  assumed  that 

the 

combined  data  and  acknowledgement  traffic  still  forms  a  Poisson 
process.  In  general,  the  acknowledgement  packets  have  different 
lengths  from  the  data  packets  and  nodal  delays  for  combined  data  and 
acknowledgement  traffic  are  determined  by  an  average  packet  length 
1/ u  '  ,  given  by 

1/V  *  y  (1/w  +  l/uack)  . 


The  total  average  message  delay  from  the  output  of  the  message 
generator  back  to  its  input,  subject  to  the  present  assumptions,  is 
given  by 


T 


1 


+  *  2/"c  +  2ka  +  h  ■ 


Unfortunately,  this  total  average  message  delay  cannot  be 
normalized  In  a  useful  manner,  as  was  the  case  for  Experiment  One, 
and  so  results  are  plotted  for  several  typical  cases.  Additional 


curves  can  be  obtained  easily  for  other  sets  of  parameter  values. 
Values  chosen  for  the  curves  are  as  follows: 
link  capacity  =  1200  bits/sec. 

^  uack  -  200  blt* 

l/u  -  500,  1000,  1500,  2000  bits/sec. 


Figures  3.4.2  through  3.4.5  give  average  message  delay  versus  message 
arrival  rate  (throughput)  for  the  sequence  of  message  lengths  listed. 
The  curves  also  show  average  message  delay  for  the  same  conditions 
without  acknowledgement  traffic.  Note  that,  as  would  be  expected, 
the  effect  of  acknowledgements  is  reduced  as  message  length  is 
increased  for  fixed  acknowledgement  packet  length. 


12.0 


NUMBER  OF  MESSAGES  PER  SECOND.  * 

FIGURE  3.4.2.  AVERAGE  MESSAGE  DELAY  VERSUS  THROUGHPUT  FOR 
MESSAGE  LENGTH  =  500  BITS,  C  =  1200  BITS/SEC.,  KA  =  K 
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FIGURE  3.4.3.  AVERAGE  MESSAGE  DELAY  VERSUS  THROUGHPUT 
MESSAGE  LENGTH  =  1000  BITS,  C  =  1200  BITS/SEC.,  I 


36.0 


FIGURE  3.4.4.  AVERAGE  MESSAGE  DELAY  VERSUS  THROUGHPUT  FOR 
MESSAGE  LENGTH  =  1500  BITS.  C  =  1200  BITS/SEC.,  K»  -  I 


NUMBER  OF  MESSAGES  PER  SECOND,  X 

FIGURE  3.4.5.  AVERAGE  MESSAGE  DELAY  VERSUS  THROUGHPUT  FOR 
MESSAGE  LENGTH  »  2000  BITS,  C  *  1200  BITS/SEC.,  KA  *  I 


3.5  Experiment  Three 


This  Experiment  uses  the  three  node  configuration  of  Figure 
3.2.1b,  but  differs  from  Experiment  One  in  that  in  this  case  the 
network  is  closed.  The  traffic  generator  is  set  up  for  closed  system 
operation  in  that  it  replaces  every  exiting  packet  immediately  with  a 
new  input  packet.  The  number,  N,  of  messages  circulating  in  the 
system  is  a  parameter  of  the  Experiment. 

Figure  3.5.1  gives  a  queueing  model  for  this  Experiment.  The 
model  is  analyzed  under  two  sets  of  assumed  conditions.  Under  the 
assumption  that  the  message  lengths  at  each  node  are  Independent  and 
exponentially  distributed,  it  is  possible  to  obtain  an  analytic 
solution  for  average  message  delay  through  the  system.  The  result, 

T '  sr  • 

is  derived  in  Appendix  3.1. 

For  assumptions  other  than  exponential  distribution  of  message 
lengths,  it  does  not  seem  to  be  possible  to  obtain  an  analytic  result 
for  average  message  delay.  However,  if  the  exponential  message 
length  assumption  is  replaced  by  the  assumption  that  message  lengths 
are  fixed  and  non-random,  which  is  a  better  approximation  to  the  real 
system,  it  is  possible  to  obtain  useful  lower  bounds  on  average 
message  delay. 


09 


** 


The  lower  bound  is  obtained  as  follows:  Consider  the  case  where 
only  one  message  circulates  (i*e.  N*i),  then  the  messge  delay,  T,  is 
3/MC,  neglecting  nodal  processing  delays.  As  the  number  of  messages 
increases,  there  will  be  some  number  such  that  at  least  one  of  the 
queues  will  become  congested  so  that  its  output  link  is  never  empty, 
and  hence,  for  this  node, 

x  =  uC 


Using  Little's  equation,  the  total  average  message  delay  is 
always  given  by 


T  =  N/X 


where  X  is  the  traffic  for  a  single  loop  network. 
When  X  =  yC,  T  is  given  by 
T  =  N/uC 


and  this  then  becomes  an  asymptotic  result - 

The  two  asymptotes  3/  pC  and  N/  yC  intersect  at  N»3,  and 
together  the  curves  form  a  lower  bound  on  average  message  delay.  The 
lower  bound  and  the  result  for  exponentially  distributed  message 
lengths  are  plotted  for  message  lengths  of  500,  1000,  1500,  and  2000 
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bits  in  Figures  3.5.2  through  3.5.5. 


3.6  Experiment  Four 

This  experiment  uses  the  two  node  configuration  of  Figure 
3.2.1a,  differing  from  Experiment  Two  in  that  the  network  is  closed. 
The  new  Experiment  also  differs  from  Experiment  Three  since 
acknowledgements  mix  and  queue  with  data  traffic.  Like  Experiment 
Three,  the  traffic  generator  assures  that  N  messages  are  circulating 
in  the  network,  where  N  is  a  variable  parameter. 

Figure  3.6.1  gives  a  queueing  model  for  Experiment  Four 
including  the  effect  of  acknowledgement  traffic.  The  parameter  u' 
is  computed  in  the  same  way  as  for  Experiment  Two  and  is  given  by 

W  -  7  0/»  *  I/***) 


An  analysis  of  the  queueing  network  of  Figure  3.6.1  is  given  in 
Appendix  3.2,  assuming  exponentially  distributed  service  times.  The 
mean  average  message  delay  is  determined  to  be  given  by 

T  =  (n  Y»+HN+2)X  +  Cl\ 
\wC/\NX-(N+1)X2+XN+1  / 

where  X  ■  y‘/2y  .  if  N  is  much  larger  than  1,  however,  the 
expression  for  average  message  delay  reduces  to  T  9  N/y'C  , 
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NUMBER  OF  MESSAGES,  N 

FIGURE  3.5.2.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  -  500  BITS.  C  -  1200  BITS/SEC..  K  -  0. 


packet  lengths 
follow  an 


FIGURE  3.5.3  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  =  1000  BITS.  C  =  1200  BITS/SEC.,  K  =  0. 


deliy  « turning 


FIGURE  3.5.4.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK 
FOR  MESSAGE  LENGTH-  1500  BITS.  C  =  1200  BITS/SEC.. 
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NUMBER  OF  MESSAGES,  N 

FIGURE  3.5.5.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  =  2000  BITS,  C  =  1200  BITS/SEC.,  K  =  0. 


and  the  curves  plotted  from  the  exact  expression  show  that  this 


expression.  In  fact,  holds  reasonably  well  for  all  values  of  N. 

Lower  bounds  on  the  average  message  delay  for  the  network  of 
Figure  3.6.1,  assuming  constant  message  lengths,  can  be  obtained  in 
the  same  manner  as  for  Experiment  Three. 

The  case  of  N«1  is  again  considered  first,  and  the  resulting 
average  message  delay  depends  on  the  relative  priorities  assigned  to 
acknowledgement  and  data  traffic  on  the  channel  from  Node  B  to  Node 
A.  The  results  are  as  follows: 

data  has  priority:  T  -  3/  y C 

ack  has  priority: 

T  -  3/  UC  +  l/uack  C 
neither  has  priority: 

T  -  3/  y  C  +  1/2  uack  C  . 


At  the  extreme  of  large  N,  the  queues  which  become  congested 
first  must  be  identified.  Considering  the  expression  for  1/y'  shows 
that 

1/y  <  2/ y ' 


and  hence, 


X/yC  <  2X/y‘C 
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It  follows  that  congestion  occurs  first  at  those  queues  for  which 


average  message  length  is  1/p1  .  At  these  queues  X  approaches 
p'  C/2,  for  which  value  T  is  given  by 

I  =  2N/ti‘C  . 


Assuming  that  data  has  priority  over  acknowledgements,  the 
piecewise  linear  bound  for  T  becomes 


T  = 


'  3/uC 
, 2N/ u ' C 


N  *  3u ‘/2 
N  >  3u  72 


Figures  3.6.2  through  3.6.5  each  give  three  curves  for  average 
message  delay  versus  N,  namely:  the  curves  obtained  assuming 
exponential  message  length  distributions,  the  piecewise  linear  bound 
developed  above  and  a  curve  which  ignores  the  effect  of 
acknowledgements,  (derived  as  in  Experiment  Three).  The  Figures  use 
message  lengths  of  500,  1000,  1500  and  2000  bits. 


3.7  Experiment  Five 

Experiment  Five  is  designed  to  evaluate  the  finite  storage 
model.  To  focus  on  the  finite  storage  effect,  it  seems  reasonable  to 
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NUMBER  OF  MESSAGES,  N 

FIGURE  3.6.2.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  =  500  BITS,  C  =  1200  BITS/SEC.,  K  =  0. 


delay  curve 


FIGURE  3.6.3.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  -  1000  BITS.  C  -  1200  BITS/SEC.,  K  =  0. 


FIGURE  3.6.4.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  -  1500  BITS,  C  =  1200  BITS/SEC.,  K  -  0. 
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FIGURE  3.6.S.  AVERAGE  MESSAGE  DELAY  VERSUS  NUMBER  IN  NETWORK  FOR 
MESSAGE  LENGTH  »  2000  BITS,  C  «  1200  BITS/SEC.,  K  -  0. 


exclude  acknowledgements.  Thus  the  three  node  configuration  of 
Figure  3.2.1  can  be  used  if  the  buffers  required  by  acknowledgements 
are  ignored  or  either  of  the  two  or  the  three  node  configurations  can 
be  used  with  acknowledgements  deleted  from  the  physical  system. 

As  noted  in  discussing  the  analytical  approach,  it  is  desirable 
to  keep  the  number  of  network  states  small  so  as  to  make 
visualization  possible  in  a  three  dimensional  model  and  to  illustrate 
principles  without  the  requirement  for  extensive  numerical 
calculation. 

In  the  physical  network,  the  requirement  is  to  have  an  extremely 
small  packet  storage  capacity  at  each  node.  This  can  be  accomplished 
by  having  very  long  packets  (so  that  each  requires  a  large  number  of 
buffers)  or  by  artificially  reducing  the  number  of  available  buffers 
at  each  node. 

It  is  assumed  that  packets  which  arrive  from  the  traffic 
generator  have  a  Poisson  distribution  and  are  blocked  from  entering 
the  network  when  all  of  the  buffers  at  Node  A  are  full.  It  is 
further  assumed  that  packets  in  transit  through  the  network  are 
dropped  if  the  node  to  which  they  are  addressed  has  its  buffers  full. 
It  is  also  assumed  that  no  storage  space  is  pre-allocated  for  any 
specific  output  link. 

The  analysis  is  carried  out  for  a  two  node  network  with  storage 
available  for  two  packets  at  Node  A  and  only  one  packet  at  Node  B.  A 
queueing  model  for  the  network  is  given  in  Figure  3.7.1.  The 
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analysis  could  be  extended  relatively  easily  to  storage  for  a  few 
more  packets;  however,  practical  storage  capacities  would  require 
extensive  numerical  computation. 

With  the  stated  assumptions  and  flow  control  policies,  the 
finite  Harkov  chain  for  the  network  shown  in  Figure  3.7.2  can  be 
developed.* 

The  balance  equations  and  the  normalization  equation  can  be 
solved  for  the  state  probabilities.  Table  1  in  Appendix  3.3  gives 
the  numerical  results  for  several  values  of  p  =  X/  pC. 

From  the  state  probabilities,  the  blocking  probabilities  Pgj  » 
Pg2  and  Pg-j  may  be  evaluated;  these  give  the  probability  of  an 
external  packet  being  blocked  or  a  transit  message  being  dropped. 

The  throughput,  Y  ,  of  the  network  may  be  evaluated  to  obtain: 

3 

y  ■  a  n  (1-PR,) 

i=l 

or 

y  =  X  (1-PB1)(1-PB2)(1-PB3>  • 


*Note  that  more  assumed  storage  and  hence, 
more  states  would  result  in  an  excessively 
complicated  diagram  for  the  Markov  chain. 


For  this  network,  PB1  “  P  B3  * 

Figure  3-7.3  gives  the  throughput  versus  offered  load  for  this 
network  under  the  stated  assumptions  and  flow  control  policies  for 
uC  »  10,  an  arbitrary  value.  Figure  3.7.4  shows  how  the  average 
number  of  packets  in  the  network  varies  with  p  ,  the  utilization 
factor. 

A  network  such  as  that  considered  does  not  give  a  product-form 
solution,  i.e.,  local  balance  equations  cannot  be  written  for  the 
Markov  chain.  Hence,  no  known  closed  form  expression  can  be  written 
for  the  state  probabilities.  Since  the  Liu  Reduction  method  is 
primarily  advantageous  in  cases  for  which  the  product-form  solution 
applies,  it  is  not  used  here. 

In  reviewing  the  analysis  of  Appendix  3.3,  note  that  to  obtain 
the  throughput,  only  the  blocking  probabilities  need  to  be  evaluated. 
For  this  particular  network,  eight  of  the  eleven  state  probabilities 
are  involved  in  the  blocking  probabilities.  However,  as  the  storage 
sizes  N  ^  and  N  ^  increase,  the  relative  number  of  state 
probabilities  required  decreases.  For  cases  when  the  product-form 
solution  does  not  apply,  it  is  generally  computationally  easier  to 
calculate  the  state  probabilities  by  matrix  Inversion,  or  by  a  sparse 
matrix  technique,  than  to  use  the  Liu  Reduction  Method. 


FIGURE  3.7  3.  THROUGHPUT  VERSUS  OFFERED  LOAD  CHARACTERISTIC  <*jC  =  10). 
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APPENDIX  3.1:  ANALYSIS  OF  CLOSED  TANDEM  NETWORK 


WITH  IDENTICAL  SERVERS 


Consider  the  situation  where  N  packets  are  circulating  among  K 
queues  which  are  connected  in  tandem.  The  service  times  of  each 
server  are  exponentially  distributed  with  mean  1/  yC. 


It  can  be  easily  shown  that  the  total  number  of 
ensuing  Markov  chain  is 


/ N+K-l 
V  K-l 


states 


in  the 


Taking  any  arbitrary  queue  to  be  the  first  queue,  the  number  of 
states  with  this  queue  empty  is 

/ N+K-2 \ 


K-2 


Such  closed  queueing  networks  give  a  "product-form"  solution,  and 
since  each  queue  has  identical  packet  arrival  and  service  rates, 


«">  <ar) 1 


with  k^  +  .  • .+  k^  ■  N,  G(N)  a  normalization  constant  and  X  the 
unknown  arrival  rate. 
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Clearly,  all  states  are  equally  likely.  Hence  the  probability 


of  queue  1  being  empty  is 


K-l 

N+iTT  • 


Hence  the  probibility  of  queue  1  being  empty  Is 

1  ■  NTibr  =  N/^N+K-1)  • 

The  arrival  rate  to  each  queue,  equivalent  to  network  throughput,  Is 
then  given  by 

X  =  uCN/(N+K-l)  . 


Using  Little's  Law,  the  total  average  message  delay,  T,  in  traversing 
all  queues  is  given  by 

T  =  N/X  =  (N+K-l )/yC  . 
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APPENDIX  3.2:  ANALYSIS  OF  THE 


CLOSED  NETWORK  OF  FIGURE  3.6.1 


Consider  such  a  general  network  with  N  packets  circulating  among 
K  Identical  queues  (K-l  nodes).  Due  to  the  presence  of 
acknowledgements,  all  queues  except  the  final  “output”  queue  have  a 
service  rate  u'C  and  input  traffic  rate  2  X.  The  output  queue  has 
input  traffic  rate  X  and  service  rate  p  C. 


As  for  the  network  of  Figure  3.5.1,  there  are 


/n+k-i\ 
\  K-l  / 


states  of  which 


N+K-2\ 

K-2  / 


Have  the  output  queue  empty.  In  this  case  all  states  are  not  equally 
likely. 


The  probability,  Pj(q  ,  of  the  output  queues  being  empty  can  be 


computed  in  a  straightforward  way  to  obtain 


t 


In  the  network  under  consideration  there  are  three  queues ,  so  that 
K  ■  3  and 


P 


KO 


* - — -  =  (1-X)2(N+1) 

£  (N+l-i)(lJ  ,/2p)i  N+1-NX-2X+XN+2 

1=0 


where  X  -  u  1  /  2  y  . 

; 

I  The  throughput  of  this  network,  x  ,  is  given  by 


x  =  uC 


By  Little's  Law,  the  average  network  delay,  T,  is  given  by: 


N+2 

N_  e  N-H  -  NX  -  2X  +r 
uC  NX  -  (N+1 )X2  +  XN+2 


Since  X  <  1  always,  the  term  X  may  be  neglected  except  for  small 

values  of  N.  Hence, 

I1  +  fT-IFYN } 

for  large  enough  N  . 


UZT5T 

2N 
u 1 C 
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APPENDIX  3.3:  BALANCE  EQUATIONS  FOR  THE 
MARKOV  CHAIN  OF  FIGURE  3.7.2 


The  balance  equations  are : 

X  p001  +  m2  p110  +  m2  pm  =  p101 

UZ  p010  +  m3  p0Q2  s  (X+y3^  p001 

y3  p001  =  X  p000 

x  p000  +  m3  p101  =  P100 

U1  ^p100+p110^  +  w3  p011  *  (x+y2^  p010 

y3  p111  +Xp010  +  U1  p210  +  y1  p200  =  <Wx)  pno 
A  p11q  *  (u^uz)  P210 

A  P100  +  u2  P210  =  U1  P200 

yi  pioi  +  yi  pm  =  (x+tJ2+p3^  pon 

y2  pon  =  y3  p002 
a  p011  *  (u1+v2+y3^  pm  * 

where 

Z!  pijk  *  1  , 
all  1,j,k 
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the  P1-jk  are  the  stationary  probabilities  of  having  k  messages 
in  queue  1,  etc.,  and  y..  in  these  equations  den-'tes  the  C  from 

the  text. 

The  solutions  of  these  equations  for  specified  values  of 

p  =  x/y  are  given  in  Table  1.  Additional  quantities  required  are: 

Pg-j  =  probability  of  blocking  at  queue  1 
=  p101  +  p210  +  p200  +  p002  +  plll 
=  PB3 

PB2  =  p  21  o  +  pni  +  pon  +  pno  +  poio 

N  =  average  number  of  packets  in  the  network 

*  L  »1Jk 

all  i,j,k 

p001  +  p100  +  p010  +  2  (p101  +  p110  +  p200  +  p002 
+  p011}  +  3  (plll  +  p210) 


Note  that  with  the  restrictions  -  2,  Np  -  1,  there  are  12 
possible  states  from  this  network.  However,  only  eleven  of  these  are 
reachable;  state  (012)  cannot  be  reached  for  any  of  the  others,  and 
hence  PQ]2  “  °* 


TABLE  1.  State  probabilities  for  different  p  =  x/u  . 


X/p 

" 

0.05 

0.10 

0.15 

0.20 

0.25 

0.30  j 

O 

O 

o 

a 

0.789994 

0.639627 

0.527388 

0.441902 

0.375144 

0.322037  I 

p210 

0.001061 

0.003668 

0.007234 

0.011398 

0.015927 

0.020664 

pni 

0.000009 

0.000063 

0.000190 

0.000403 

0.000708 

0.001109  j 

PJ10 

0.040933 

0.068422 

0.087176 

0.100017 

0.108733 

0.114503  | 

P1J0 

0.058772 

0.094401 

0.115968 

0.128675 

0.135658 

0.138876 

p200 

0.004000 

0.013108 

0.024629 

0.037133 

0.049842 

0.062327 

p001 

0.039500 

0.063933 

0.079108 

0.088381 

0.093786 

0.096611 

P1 10 

0.042437 

0.073361 

0.096454 

0.113981 

0.127417 

0.137762 

p011 

0.000542 

0.001903 

0.003798 

0 .006039 

0.008499 

0.011092 

°002 

0.00C542 

0.001903 

0.003798 

0.006039 

0.008499 

0.011092 

°1J1 

0.022211 

0.039909 

0.054255 

0.066030 

0.075786 

0.083927 

PB1 

0.027823 

0.058651 

0.090110 

0.121003 

0.150762 

0.179119 

PB2 

0.084982 

0.147417 

0.194852 

0.231838 

0.261284 

0.285130 

y/x 

0.864809 

0.755506 

0.666582 

0.593509 

0.532766 

0.481712 

N 

0.281879 

0.495317 

0.670392 

0.810920 

0.928168 

1.027709  | 

NX 

Y 

. 

0.325944 

0.659580 

1 .005716 

1 .366315 

1.742168 

1 

2.133451  | 

i 

L 
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4.  EXPERIMENTAL  RESULTS 


4.1  Int  roduct ion 

In  this  section  the  experimental  results  of  the  proposed 
experiments  described  In  Section  3  are  presented.  The  network  was 
connected  In  a  numb  ~  of  configurations  and  the  performance 
monitoring  techniques  described  In  Section  5  were  used  to  collect 
information  about  message  delays  in  the  system.  The  Nova  CS-20 
system  was  used  to  generate  traffic  to  the  network  in  different 
statistical  distributions.  Because  of  hardware  limitations  of  the 
CS-20  serial  i/o  interface  the  rate  at  which  the  network  could  be 
made  to  accept  data  was  very  limited  and  experimental  results  was 
limited  to  the  lower  part  of  the  curves  in  Section  3. 


4.2  Explanation  of  Hardware  Limitations 

The  DTR  (data  terminal  ready) 

And  DSR  (data  set  ready)  interface  lines  on  the  CS-20  computer 
are  not  connected  to  the  data  section  of  the  serial  interface.  This 
forced  the  CS-20  CPU  to  monitor  and  control  these  lines  and  send  the 
proper  control  signals  to  the  data  section.  These  lines  are  not 
available  as  an  interrupt  so  that  the  control  and  monitor  process 
must  be  maintained  at  a  rapid  interval  to  assure  proper  handshake 
operation.  If  the  DTR  from  the  network  was  lowered  and  returned 
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(message  recieved  by  network)  at  a  time  when  the  CS-20  was  not 


available  to  monitor  the  operation  then  the  CS-20  would  assume  that 
the  network  was  not  ready  to  accept  more  data  and  would  stop 
sending.  The  network  would  assume  that  the  ^8-20  had  recieved  the 
handshake  protocol  and  would  wait  for  the  next  character.  This 
would  stop  all  operation  of  the  traffic  generator  and  the  traffic 
program  would  need  to  be  cleared  and  restarted.  Several  software 
techniques  where  developed  to  detect  when  the  system  had  missed  the 
hardware  protocol  and  automatically  clear  the  hardware  and  continue 
operation.  This  software  solution  kept  the  system  from  terminating 
operation  but  required  significant  amount  of  time  and  resulted  in 
the  loss  of  one  message. 

The  final  system  available  to  send  traffic  to  the  network  was 
therefore  limited  in  speed  and  would  tend  to  lose  messages  as  the 
speed  limit  was  approached.  This  problem  would  tend  to  emphasis  the 
importance  of  handshake  protocol  lines  in  the  network  serial 
interfaces . 


4.3  Low  Speed  Network  Data 

The  system  was  connected  in  different  configurations  and  loaded 
with  message  traffic  of  different  bit  lengths.  The  system  was 
configured  with  the  traffic  generator  with  one  to  three  nodes. 
These  configurations  were  run  with  and  without  local 
acknowledgments.  Message  lengths  varied  from  10  to  60  characters 
(100  to  600  bits). 


99 


u . 


i.fcisi  ... 


The  message  data  was  collected  as  a  table  of  departure  times 
and  arrival  times  of  each  message.  Figure  4.1  shows  a  typical  data 
table  where  the  first  column  represents  the  number  of  the  message 
being  transmitted  or  recieved,  the  second  column  represents  the 
number  of  bytes  in  a  departing  message,  the  third  column  represents 
the  time  of  departure  in  counts,  and  the  fourth  column  represents 
the  time  of  arrival  of  the  return  message  in  counts.  One  count  in 
the  departure  and  arrival  column  represents  a  time  interval  of 
1.8432  millisecond.  Time  delay  can  be  determined  by  calculating  the 
diffenence  between  departure  and  arrival.  For  example  the  time 
delay  for  message  one  equal  1870  -  859  -  1011  counts  or  1.86 
seconds. 

The  data  obtained  in  the  low  speed  experiments  compared  well 
with  theoretical  estimate  as  shown  in  the  comparison  values  in 
Figure  4.2.  This  data  was  obtained  by  sending  messages  varing  in 
length  from  10  to  210  at  300  baud  through  the  system  shown  in  Figure 
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FIGURE  4.1  DATA  TABLE 
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NO.  OF  BYTES  PER 
MESSAGE 


MEASURED 

DELAY 


THEORETICAL 

DELAY 


10 

2.7 

SEC  . 

2.7 

SEC  . 

20 

4.3 

SEC  . 

4 . 5 

SEC  . 

30 

5.7 

SEC  . 

5.8 

SEC  . 

40 

8.4 

SEC  . 

8  .  5 

SEC  . 

60 

9.8 

SEC  . 

9.8 

SEC  . 

80 

12.7 

SEC  . 

12.5 

SEC  . 

110 

16.9 

SEC  . 

16.9 

SEC  . 

160 

24 . 0 

SEC  . 

24 . 0 

SEC  . 

200 

31  .2 

SEC. 

31.0 

SEC  . 

FIGURE  4.2  EXPERIMENTAL  DATA  EVALUATION 
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5.  Performance  Monitoring 


5.1  Introduction 

In  order  to  adequately  evaluate  the  validity  of  the  computer 
models  presented  in  the  previous  sections,  it  was  necessary  to 
collect  data  regarding  the  actual  performance  of  these  networks  and 
to  compare  that  data  with  the  performance  as  predicted  by  the 
models.  Certain  types  of  data  regarding  the  network  performance 
could  be  monitored  externally  by  monitoring  the  traffic  going  into 
the  network  and  the  traffic  coming  back  out  of  the  network.  For 
example,  total  network  loop  delay  could  be  found  by  generating  a 
message  which  is  routed  back  to  the  original  center  for  which  total 
time  between  the  entrance  of  the  message  into  the  network  and  its 
return  is  measured.  Some  types  of  performance  data  could  not  be 
measured  externally  and  could  be  obtained  only  by  monitoring  events 
inside  the  communication  nodes.  An  example  of  this  type  of  data  is 
the  number  of  messages  queued  at  any  one  particular  node. 

There  are  several  techniques  that  can  be  used  to  retrieve 
internal  performance  data  from  the  microprocessor  communication 
nodes.  One  approach  is  to  add  portions  to  the  communication  node 
software  so  that  the  communication  node  continually  sends  out 
information  regarding  its  status.  This  software  monitoring  approach 
has  the  advantage  of  requiring  little  or  no  additional  hardware  to 
achieve  performance  monitoring,  but  has  the  disadvantage  of  requiring 
a  certain  amount  of  software  time  which  distracts  from  the 
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performance  of  the  network,  thereby  affecting  the  performance  of  the 
system. 

An  approach  that  tends  to  minimize  the  interaction  of  the 
communication  process  and  the  monitoring  process  is  the  use  of  a 
combination  of  hardware  and  software.  With  this  technique,  small 
software  routines  with  fast  execution  times  are  placed  in  the  major 
software  loops  to  pick  up  key  information  from  the  communication 
process  without  slowing  down  the  process  significantly.  Hardware 
added  to  the  system  picks  up  this  key  information  and  transmits  it  to 
a  central  collection  system.  This  is  a  compromise  system,  where 
hardware  is  added  to  minimize  the  effect  of  the  software  execution 
time,  but  does  not  totally  eliminate  the  impact  of  the  performance 
monitoring  task  on  the  communication  task. 

A  third  method  uses  hardware  added  to  communication  processors 
to  measure  the  node  performance  without  any  intervention  in  the 
communication  software.  This  technique,  although  more  expensive,  has 
the  advantage  of  being  totally  non-in  as ive  and  therefore  will  not 
affect  the  system's  performance  in  its  attempt  the  measure  it.  In 
this  research  project,  completely  non-invasive  hardware  type 
monitoring  was  used  to  measure  network  performance  to  guarantee  that 
the  measuring  process  would  in  no  way  affect  the  performance  of  the 
network  itself. 


5.2  Monitoring  Approach 


The  communication  processors  consist  of  the  CPU,  the  ROM  memory, 
containing  the  processor  control  software,  the  RAM  memory,  containing 
the  message  data  with  its  associated  queues  and  stack  pointers  and 
four  serial  I/O  ports  for  transmitting  and  receiving  data. 

If  care  is  taken  in  designing  the  software  for  the  communication 
processing,  selected  memory  locations  within  the  random  access  memory 
will  contain  sufficient  information  to  accurately  reflect  the 
performance  of  the  comnunication  node  at  any  particular  time.  These 
locations  include  the  current  values  of  stack  pointers,  queue  links, 
source  and  destination  codes,  etc. 

The  basic  approach  for  implementing  a  hardware  monitor  for  the 
communication  network  is  to  use  a  two  port  RAM  memory  with  one  of  the 
two  ports  connected  to  the  communication  processor  CPU  for  the  normal 
storage  and  retrival  of  message  data  and  its  associated  buffer 
information.  The  second  port  of  the  memory  would  allows  an 
additional  central  processing  unit  to  access  the  same  memory  space 
and  retrieve  information  necessary  for  performance  monitoring  without 
interrupting  the  process  of  the  communication  CPU.  The  speed  of  the 
two  port  memory  is  sufficiently  fast  so  that  the  central  processors 


can  access 

the  memory 

with  no 

degradation  in 

speed  to 

the 

communication 

process . 

In 

order 

to  maintain  the 

speed 

and 

versatility 

required 

by 

the 

communication 

process, 

the 

interconnection  between  the  communication  central  processor  and  the 
port  on  the  RAM  memory  is  constrained  to  be  the  standard  memory  bus 
interface  used  by  the  communication  central  processor. 
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The  central  processor  used  for  the  performance  monitoring  of  the 
monitor  CPU  does  not  require  as  rigid  a  time  access  to  the  memory  and 
could  be  forced  to  wait  several  memory  cycles  for  access  to  the  RAH 
memory.  Because  of  this  flexibility,  two  techniques  are  employed  and 
implemented  from  the  monitor  central  processor  to  the  other  port  of 
the  RAM  memory.  The  first  technique  is  a  time-shared  data  bus  where 
both  central  processors  are  on  the  same  memory  bus  and  take  turns 
accessing  the  memory.  This  technique  requires  time  coordination 
between  the  two  central  processors  and  a  memory  that  is  sufficiently 
fast  to  be  accessed  by  both  central  processors  without  loss  of  time 
to  either  processor.  The  second  technique  uses  an  I/O  type  interface 
between  the  monitor  CPU  and  the  memory.  The  monitor  central 
processor  sends  out  a  request  for  a  given  memory  location  and  the 
interface  to  the  memory  retrieves  the  contents  of  that  location  when 
possible  without  disturbing  the  communication  CPU.  The  interface 
flags  the  CPU  when  the  data  is  ready  and  the  monitor  CPU  reads  in  the 
data  through  an  I/O  type  input  port. 

In  the  following  two  sections  each  of  these  techniques  will  be 
described  in  detail,  including  both  their  advantages  and 
disadvantages 


5.3  Time-Shared  Bus  Interface  For  Monitor  CPU 

As  mentioned  above,  one  technique  for  interfacing  the  monitor 
CPU  to  the  two  port  RAM  memory  is  to  connect  the  monitor  CPU  to  the 
standard  memory  bus  and  to  time  share  the  memory  bus  availaility 
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between  the  communication  CPU  and  the  monitor  CPU.  The  normal 
operation  of  the  central  processor  is  controlled  by  a  two  phase  clock 
as  shown  as  01  and  02  in  Figure  1.  The  memory  is  accessed  through 
tri-state  buffers  by 

Ending  the  valid  memory  address  line  with  the  02  line.  In  this 
way,  during  the  01  cycle  all  of  the  memory  address  and  access  lines 
are  in  the  open  tri-state  configuration;  therefore,  a  second  CPU 
cou  '  ■}  freely  access  the  memory  if  it  is  insured  that  all  accesses 
were  permitted  only  during  the  01  cycle.  This  can  be  done  by  using 
one  clock  to  control  both  the  communication  and  the  monitor  CPU  but 
reversing  the  phases  on  the  monitor  CPU,  as  illustrated  in  Figure  2. 

However,  this  very  simple  clocking  scheme  produces  certain 
timing  problems  because  the  02  pulse  is  used  as  a  memory  strobe  and 
the  memory  address  lines  may  not  have  time  to  stablize  prior  to 
memory  strobe.  To  eliminate  these  timing  problems,  a  slightly  more 
complex  clocking  scheme  as  shown  in  Figure  3  is  used.  In  this 
scheme,  the  bus  enable  pulses  that  control  which  CPU  can  access  the 
bus  is  separated  from  the  02  pulse  to  permit  additional  settling  time 
between  enablng  the  bus  and  the  memory  strobe.  The  schematic  of  the 
central  processor  before  modification  for  the  two  port  memory  is 
shown  in  Figure  4.  The  central  processor  with  the  modifications  are 
shown  in  Figure  5. 

The  synchronous  clock  waveforms  for  01,  02  and  the  bus  enable 
lines  were  implementd  by  blasting  the  on  and  off  bit  patterns  into 
the  first  sixteen  consecutive  addresses  of  a  read-only  memory.  The 


FIGURE  5.3  Modified  Dual  CPU  Clock 


ROM  address  lines  are  attached  to  a  16-bit  counter  which  is 
continually  clocked  at  sixteen  times  the  effective  01,  02  clock  rate. 
The  ROM  is  therefore  continually  swept  through  the  first  sixteen 
addresses  and  produces  all  four  of  the  waveforms  needed  for  the  CPU 
clocks.  4  functional  block  diagram  of  this  clocking  system  is  shown 
in  Figure  6.  This  circuit  was  added  to  the  system  by  constructing  a 
new  printed  circuit  board  which  contains  the  clock  circuitry  plus  an 
additional  RS232  serial  port,  which  is  used  by  the  monitor  CPU  to 
transmit  performance  data.  The  circuit  diagram  of  this  card  is  shown 
in  Figure  7. 

The  communication  node  hardware  with  performance  monitoring  is 
comprised  of  a  communication  CPU  to  control  the  communication 
process,  a  monitor  CPU  to  control  the  monitoring  process,  a  ROM 
memory  for  the  communication  and  monitoring  programs,  a  R4M  memory 
for  message  data,  four  serial  I/O  ports  of  communication  input  and 
output,  one  serial  port  for  performance  data  transmission  and  the 
time-sharing  clock  circuitry.  These  are  distributed  on  printed 
circuit  cards  as  illustrated  in  Figure  8. 


5.4  I/O  Based  Interface  For  The  Monitor  CPU 

4s  discussed  in  Section  II,  another  possibility  for  interfacing 
the  monitor  CPU  to  the  two-port  memory  is  to  interface  the  CPU  by 
means  of  standard  parallel  I/O  ports.  With  this  technique,  the 
monitor  CPU  sends  out  the  address  of  the  memory  location  it  wants  to 
read  on  sixteen  I/O  lines  and  sets  a  flip  flop  Indicating  that  it  Is 


113 


FIGURE  5.6  Clock  Implementation 
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5.7  CIRCUIT  DIAGRAM 


requesting  data.  Control  circuitry  interfaced  to  the  memory  bus  then 
detects  the  next  available  time  that  the  memory  is  not  being  accessed 
by  the  communication  CPU,  retrieves  the  contents  of  the  specified 
memory  location  and  places  it  in  a  holding  buffer.  The  control  logic 
re-sets  the  flag  flip  flop  as  an  indication  to  the  monitor  CPU  that 
the  data  has  been  retrieved.  The  monitor  CPU  then  reads  the  holding 
register  using  normal  parallel  I/O  lines,  k  functional  block  diagram 
of  this  system  is  shown  in  Figure  9. 

k  prototype  system  using  a  6502  microprocessor  as  shown  in 
Figure  10  was  built  and  successfully  operated  with  the  communication 
6800  CPU  and  memory  board. 

5.5  Comparison  of  the  Time-Sharing  Interface  and  I/O  Interface 
Techniques 

Both  the  time-shared  interface  and  the  I/O  interface  for  the 
monitor  CP  were  successfully  built  and  tested  v>ith  th«>  communication 
6800  central  processor  and  two  port  memory.  The  advantages  and 
disadvantages  of  these  techniques  will  influence  the  selection  of 
which  technique  will  be  used  for  a  particular  application.  The 
following  is  a  list  indicating  the  advantages  of  each. 

Time-Shared  Bus  Interface  Advantages: 

1.  Very  little  specialized  hardware  is  required  for  implementation. 


ADDRESS 

REGISTER 


CPU 


FIGURE  5.9  Prototype  System  Block  Diag 


2.  The  monitor  CPU  access  to  the  two  port  memory  is  extremely  fast 
since  it  never  has  to  wait  on  the  communication  CPU  for  access* 


Software  in  the  monitor  CPU  can  be  minimized  since  a  single 
instruction  is  adequate  for  reading  the  two  port  memory. 

1/0  Interface  Advantages 

1.  Does  not  require  as  fast  a  memory  as  the  time  shared  techniques. 

2.  Can  be  used  between  different  types  of  CPU. 
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APPENDIX  A 


Communication  Software  Flow  Charts 
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BUFFER  STATUS  TABLE 


Relative 

Location 


0 


NOBUF 


Name  Description 

BST  -One  byte  per  buffer 

-Buffer  is  a  non-zero  number 

-0  =  Not  in  use 

-FF  =  In  use  by  CST 

-FE  =  Local  acknowledgement  packet 

-FD  =  Source  acknowledgement  packet 

-FC  =  Reserved  for  monitor 

-ELSE  =  Message  number 
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get  the  message  number 
which  is  waiting  to  be 
auened 


Z  HI,  RTI 
carry =1,=?>  B=message  » 
carr y=0,=>B=0  $ 
v  no  message 


queue  a  message  from 
message  status  table  to 
data  message  output 


Z  El,  RTS 
carry*  1  ,=N  job  done 
carry-2 ,  queue  full, 
l  job  not  don 


NOS  #9 


NOS  #10 


El ,  RTI 
fcarry=l ,=>  B=message  # 
carry=2fs> B=0  4 
V  no  message 


S  El,  RTI  X 

/carry  set=>time  out, 
i  B=message  # 

\arry  clear->Not  time/ 
x^"out,  Ji=0 


CHANNEL  STATUS  TABLE  DEFINITION 


Input : 


0) 

1 

Byte  - 

1) 

1 

Byte  - 

2) 

1 

Byte  - 

3) 

1 

Byte  - 

4) 

1 

Byte  - 

5) 

1 

Byte  - 

6) 

1 

Byte  - 

7) 

1 

Byte  - 

8J 

1 

Byte  - 

9) 

1 

Byte  - 

Output : 

0) 

1 

Byte  - 

1) 

1 

Byte  - 

2) 

1 

Byte  - 

3) 

1 

Byte  - 

4) 

1 

Byte  - 

S) 

1 

Byte  - 

6) 

1 

Byte  - 

7) 

1 

Byte  - 

8) 

1 

Byte  - 

9) 

1 

Byte  - 

A) 

2 

Bytes- 

C) 

1 

Byte  - 

First  Buffer  Number  (Non- Zero) 

Total  Number  of  Buffers 
Present  Input  Buffer  Number 
Location  in  Present  Buffer 
E-Flag,  0  =  Not  Found,  1  =  Found 
High  Order  LRC  (Real) 

Low  Order  LRC  (Real) 

High  Order  LRC  (Temporary) 

Low  Order  LRC  (Temporary) 

High/Low  LRC  Switch,  0  *  High,  1  =  Low 


Total  Number  of  Output  Buffer  Left 
1  =  Last  Buffer,  0  =  No  Output  Channel  Status  Table 

Number  of  Bytes  in  the  Last  Buffer 

Present  Buffer  Number 

Location  in  Present  Buffer 

E-Flag 

STX-Flag,  1  =  STX  Sent,  0  =  Not  Sent 
EXT-Flag,  1  =  ETX  Sent,  0  =  Not  Sent 
Output  LRC  High  (Not  Used) 

Output  LRC  Low 

High/Low  Switch  for  LRC 

Address  of  This  Message  Status  in  MST 

Channel  Sequence  Number 


1 


) 


I 
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PACKET  PORMAT 


Relative 

Position  Content 

00  02 
01 

01 

02 

03 

04 


02 

1  <x  <$32 

03 

0<x<  maximum 
buffer  size 

04 

x 

05 

X 

0b 

X 

07 

X 

08 

X 

00 

X 

0A 

X 

• 

X 

• 

X 

y 

X 

y  +  1 

X 

y  +  2 

X 

Description 

STX  (Start  of  message) 

Packet  type  identifier 

Source  acknowledgement 

Local  acknowledgement 

Data  Packet 

Executable  data  packet 

Total  number  of  buffers  needed  for 
this  packet 

Number  of  bytes  in  the  last  data 
packet 

Packet  origin 

Packet  destination 

Origin  message  sequence  number 

Origin  packet  sequence  number 

Local  packet  sequence  number 

Data 


Data 

High  order  LRC 
Low  order  LRC 
End  of  packet 


ETX  04 

y  =  Maximum  buffer  size  -3 
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MESSAGE  STATUS  TABLE  DEFINITION 


0)  1  Byte  -  Processing  Status: 

0  =  No  message  (this  entry  not  in  use) 

1  «  This  waiting  packet  has  been  processed  and  is  to  be 

queued  to  the  channel  specified  in  location  8 

2  *  Packet  received  without  any  detectable  LRC  error 

3  =  This  packet  is  waiting  for  local  acknowledgement 

4  =  This  packet  is  waiting  for  source  acknowledgement 

5  =  This  is  a  local  acknowledgement  packet  (highest  priority) 

6  =  This  packet  is  waiting  for  output;  after  outputting  the 

PS  should  change  to  3 

10  =  Data  packet  for  my  node 

11  =  Local  acknowledgement  has  sent  for  data  packet  for  my  node 

12  =  Local  acknowledgement  for  my  node 

1C  =  Local  acknowledgement  packet  received,  but  the  corresponding 
waiting  for  local  acknowledgement  packet  not  found 

13  =  Source  acknowledgement  for  my  node 

14  =  Data  packet  not  found 

15  =  Local  acknowledgement  has  sent  for  PS  14 
20  =  Packet  not  for  my  node 

FF  =  The  MST  entry  should  be  cleared  and  associated  buffers 
should  be  released 

1)  1  Byte  -  Local  Channel  Sequence  Number 

2)  1  Byte  -  First  Buffer  Number 

3)  1  Byte  -  Total  Number  of  Buffers 

4)  1  Byte  -  Total  Number  of  Words  in  the  Last  Buffer 

5)  2  Bytes-  Five  Minute  Timer  for  Retransmission  -  Use  Only  Lower  Byte 

7)  1  Byte  -  Total  Number  of  Times  Retransmitted 

8)  1  Byte  -  Channel  Number  (1-4)  for  Output 

9)  1  Byte  -  Input  Channel  Number 

A)  2  Bytes-  Starting  Address  of  the  Buffer  After  Processed  by  PS  2 
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NETWORK  OPERATING  SYSTEM  SUBROUTINE  GAELS 


A)  Convention 
Input : 

1)  A  -  Register  A  (always  has  the  subroutine  call 
identifier) 

2)  Interrupt  always  disabled 

3)  B  -  Register  B 

4)  X  -  Index  Register  X 

5)  RTI  -  EXIT  interrupt  routine 

6)  El  -  Enable  interrupt 

7)  Channel  Numbers  are  from  one  to  four  (1  -  4) 


Output : 

A,  B,  and  X  registers  may  or  may  not  be  changed, 
depending  on  each  subroutine  call.  Eor  details, 
please  see  the  following  assembler  listing. 


134 


VIRTUAL  MONITOR  REGISTERS 


Relative 
Locat ion 

Name 

Description 

0 

CTSC1 

Channel  table 

sequence  counter  for  channel  1 

1 

CTSC2 

Channel  table 

sequence  counter  for  channel  2 

2 

CTSC3 

Channel  table 

sequence  counter  for  channel  3 

3 

CTSC4 

Channel  table 

sequence  counter  for  channel  4 

4 

ICUC1 

Input  channel 

utilizat ion 

counter  for 

channel 

5 

ICUC2 

Input  channel 

utilization 

counter  for 

channel 

6 

ICUC3 

Input  channel 

utilization 

counter  for 

channel 

7 

ICUC4 

Input  channel 

utilization 

counter  for 

channel 

8 

OCUC1 

Output  channel 

utilization 

counter  for 

channel 

9 

OCUC2 

Output  channel 

utilization 

counter  for 

channel 

10 

OCUC3 

Output  channel 

utilization 

counter  for 

channel 

11 

OCUC4 

Output  channel 

utilization 

counter  for 

channel 

12 

MPUID 

MPU  idle  count 

13 

MS  C 

Message  sequence  counter 

1 

2 

3 

4 

1 

2 

3 

4 


VIRTUAL  MONITOR  LRROR  REGISTERS 


Relative 


,ocat  ion 

Name 

Doscr i pt ion 

0 

El 

Unrecognizable  interrupt 

1 

E2 

ACIA  input  hardware  error 

2 

E3 

Input  CST  not  found,  and  the  incoming 
character  is  not  STX 

3 

E4 

Buffer  full  (incoming  message  will  be 
dropped) 

4 

E5 

ACIA  output  hardware  error 

5 

E6 

Message  too  long  5  not  enough  buffer 
to  store  it 

6 

E7 

LRC  error  for  both  control  5  data  packet 

7 

E8 

Unrecognizable  control  packet 

8 

E9 

Message  needs  to  be  retransmitted 

9 

E10 

Messages  need  to  be  retransmitted 

10 

Ell 

Number  of  local  acknowledgements  received 

11 

El  2 

Interrupt  asserted  but  neither  input 
nor  output 

12 

El  3 

Local  acknowledgement  docs  not  exist 

13 

E14 

Unrecognizable  NOS  command 

14 

E15 

Release  wrong  buffer 

15 

E16 

MST  full,  message  tossed 

16 

El  7 

Output  portion  of  CST  does  not  exit 

17 

El  9 

Transmitter  interrupted  but  TBUSY  is 
not  equal  to  1 

18 

E20 

Unknown  packet  type 

19 

E21 

Status  of  bad  ACIA 

20 

BACIA 

Channel  number  of  bad  ACIA 

21 

BACIA+1 

1  =  input;  0  =  output 
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NLTWORK  MONITOR 


NETWORK  MONITOR,  con't 


ny  s. 

^^Data  Packet^ 
Receiving  A  Local 
"'vAcknowledgemep*' 


PS  12 

Change  Its  PS  To  FF 


/  Any  Data  n 
Packet  Not  For  My 
N.  Node?  V 


PS  20 

C^icue  It  To  The 
Appropriate  Output 
Channel 


^  Any  MST  N 
Entry  Should  Be 
^  Cleared? 


PS  *  FF 

Cleared  The  MST  Entry j 


entrant  Interrupt  Dispatch  Routine,  con't 


INTERRUPT  RECEIVER  ROUTINE 


Interrupt  Receiver  Routine,  Con't 


Interrupt  Transmitter  Routine,  con't 


Interrupt  Transmitter  Routine, 


1) Set  A  =  $95 

2) NRTS  =»  Low 

3) RTX  Receiver  Interrup 
Enabled 

4) TXD  Interrupt  Disable 


Input  Interrupt  lixit,  con't 
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1) Set  NRTS  3  Low 

2) RXD  I NT  Enable 

3) TXD  INT  Disable 


Subroutine  HANDSHAKli,  con't 


